Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[HUDI-6787] DO NOT MERGE. Test hive3 in ghactions #11398

Closed
wants to merge 42 commits into from
Closed

[HUDI-6787] DO NOT MERGE. Test hive3 in ghactions #11398

wants to merge 42 commits into from

Conversation

jonvex
Copy link
Contributor

@jonvex jonvex commented Jun 5, 2024

Change Logs

Testing hive 3 using the bundle validation setup

Impact

see if hive 3 works as expected

Risk level (write none, low medium or high below)

none

Documentation Update

N/A

Contributor's checklist

  • Read through contributor's guide
  • Change Logs and Impact were stated clearly
  • Adequate tests were added if applicable
  • CI passed

Jonathan Vexler added 30 commits May 31, 2024 14:42
@codope
squash commits
aa6e1c2
@codope
refactor a bit and add some comments
3674390
@codope
make build properly
edad863
@codope
fix some of the failing tests
b764026
@codope
revert to old impl when schema evolution enabled
6647869
@codope
disable fg reader for stupid test
b43949b
@codope
fix some failing tests
2bc441f
@codope
assigned the ports backwards
1bcbab8
@codope
verbose output bundle validation
6e5fadd
@codope
add volume to docker compose
dc4ac6f
@codope
need to lowercase the field names omg
8669cce
@codope
don't remove partition if it's listed in the schema
fdeea22
@codope
put partition cols at end of output
60fe6fa
@codope
invert filter
ad3f3d3
@codope
support no base file, only log. support read from non-hudi table
6a77480
@codope
disable for skip merge as well
51f91c7
@codope
fix non hoodie path read
1da1e76
@codope
revert setting precombine
2b1f172
@codope
fix no meta cols table
7b45027
@codope
check if no requested fields
16cff40
@codope
create empty schema properly
e115fd5
@codope
check if metadata folder exists
211bcb9
@codope
handle mor with no meta fields
73bedb4
@codope
disable reader for a test because mor seems to work different
3f412a1
@codope
delete partition column from the jobconf if it is written in the file
51a47e4
@codope
modify data schema due to partition column madness
a8020bd
@codope
remove unused import
25078f3
@codope
add some comments
c225f52
@codope
don't add partition fields when the data schema doesn't have them
069a5ad
@codope
address review feedback
185217d
@github-actions github-actions bot added the size:XL PR with lines of changes > 1000 label Jun 5, 2024
@hudi-bot
Copy link

hudi-bot commented Jun 5, 2024

CI report:

Bot commands @hudi-bot supports the following commands:
  • @hudi-bot run azure re-run the last Azure build

yihua
yihua reviewed Jun 10, 2024
View reviewed changes
Copy link
Contributor

@yihua yihua left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this testing PR still needed?

@jonvex jonvex closed this Jun 11, 2024
@yihua yihua mentioned this pull request Jun 23, 2024
Merged
4 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@yihua yihua yihua left review comments

Assignees
No one assigned
Labels
size:XL PR with lines of changes > 1000
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@jonvex @hudi-bot @yihua @codope